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ABSTRACT 

NASA Langley has conducted research in the area of helmet-mounted/head-worn displays over the past 30 
years. Initially, NASA Langley’s research focused on military applications, but recently has conducted a line 
of research in the area of head- worn displays for commercial and business aircraft. This work has revolved 
around numerous simulation experiments as well as flight tests to develop technology and data for industry 
and regulatory guidance. The paper summarizes the results of NASA’s helmet-mounted/head-worn display 
research. Of note, the work tracks progress in wearable collimated optics, head tracking, latency reduction, 
and weight. The research lends credence that a small, sunglasses-type form factor of the head-worn display 
would be acceptable to commercial pilots, and this goal is now becoming technologically feasible. The research 
further suggests that a head-worn display may serve as an “equivalent” Head-Up Display (HUD) with safety, 
operational, and cost benefits. “HUD equivalence” appears to be the economic avenue by which head-worn 
displays can become main-stream on the commercial and business aircraft flight deck. If this happens, NASA’s 
research suggests that additional operational benefits using the unique capabilities of the head-worn display can 
open up new operational paradigms. 
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1. INTRODUCTION 

A significant portion of NASA Langley’s Crew Systems branch mission is to conduct research to advance the 
state-of-the-art in flight deck interface technologies, including visual displays. The heritage of this work was tied 
closely to the Department of Defense (DoD) activities but has changed over the last two decades to focus on 
research to provide commercial flight crews with proactive, intuitive tools to conduct a safe and efficient flight. 
Since the late 1990’s, this research was driven by the White House Commission on Aviation Safety in 1997. 1,2 
While these safety initiatives firmly remain, NASA research is also closely tied to the modernization of the 
National Airspace System (NAS) known as the Next Generation Air Transportation System (NextGen). The 
goal of NextGen is to remove many of the constraints in the current air transportation system, support a wider 
range of operations, and deliver significantly increased system capacity to that of current operating levels. The 
NextGen concept for the year 2025 and beyond envisions the movement of large numbers of people and goods in 
a safe, efficient, and reliable manner. Operating concepts emerging under NextGen require new technology and 
procedures not only on the ground-side but also on the flight deck. One of the key elements is to create a NAS 
that is resilient, if not immune to the impacts of weather. 

This paper provides a high-level overview of NASA Langley’s Helmet Mounted Display (HMD) and subsequent 
Head- Worn Display (HWD) research as it relates to advancing the state-of-the-art to develop technology and 
data for industry and regulatory guidance. The work indicates direct support of proactive, intuitive tools to 
conduct a safe and efficient flight in a NextGen environment and possibly suggests that the unique capabilities of 
the head-worn display can open up new operational paradigms in commercial and business aircraft operations. 

2. BACKGROUND 

NASA Langley Research Center’s (LaRC’s) work while advancing the state-of-the-art has primarily focused on 
the human factors of HWDs, roughly categorized into four areas. 
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2.1 Research Issues 


First, NASA’s research has focused on the ergonomic or anthropomorphic issues of head-worn displays. For- 
tunately, the consumer market and its drive for small, lightweight displays for consumer applications has not 
required significant technology investment to move the state-of-the-art for commercial flight deck applications. 

The second research area is physiological. HWDs may induce a variety of visual-vestibular interactions, each 
of which can have a significant impact on the user. 

Third, perception and perceptual issues figure prominently on a list of human factors research-needs. The 
visual perception, comprehension, and understanding of HWD information depends on many parameters, not 
the least of which includes the optical performance characteristics of the HWD. In short, perception issues create 
significant challenges to match the human visual system with the human mind as it relates to the processing and 
interpretation of the visual stimuli. 

Finally, the fourth area of human factors research-needs is operational. According to Velger, 3 how much and 
what type of information and how it should be displayed represent perhaps the, . . most indeterminate and 
insufficiently defined subjects” (p. 179). Some years later, the issues associated with operational needs and how 
to meet these needs with appropriate informational display content still looms large as significant human factors 
concerns and research-needs include: 4 


• What is the appropriate amount and type of information to be displayed? 

• What is the most effective presentation of that information? 


3. HWD/HMD RESEARCH AT NASA LANGLEY 




Figure 1. Monochrome, binocular Figure 2. Monochrome, binocular 

HMD circa 1988, 9 pounds. HMD circa 1991, 6.5 pounds. 



Figure 3. Full color, binocular pro- 
totype HMD by Microvision circa 
2003, 10 pounds. 


3.1 Early HMD research 

The HMD has a theoretical existence that dates back a hundred years. However, it was not until the 1980s 
that advanced display capabilities really started to emerge where information beyond a simple gun-sight could 
effectively be employed in a head-worn device. The equipment during the 1980s was extremely technologically 
advanced but in reality, not viable for use on a flight deck (see Fig. 1). This state-of-the-art system used 
binocular optics with 80° circular oculars and a 40° stereo overlap generated by monochromatic green Cathode 
Ray Tubes (CRTs) drawing 875 scan lines each. The system employed an alternating current (AC) head- 
tracker. The head-borne weight was approximately 9 pounds, not including the active cooling provided by 
the air conditioning cooling hose. This particular HMD was not used in any published research and was the 
predecessor of the HMD used in the High Angle of attack Research Vehicle (HARV) research (see Fig. 2). 


3.1.1 High angle of attack research 

This HMD and others like it supported numerous military applications including Apache, emerging Comanche 
program requirements, and high angle-of-attack research, such as that deployed on the NASA thrust-vectored 
F-18 HARV. 5 ~ 9 LaRC’s research emphasized stereo/bi-ocular effects and off-boresight informational constructs. 

For the HARV Program, HWD research targeted the obvious emphasis of off-boresight capabilities especially 
as it applies to the extreme maneuvering envelope provided by this vehicle. NASA Langley-developed research 
HMD had a wide-freld-of-view (30° vertical by 40° horizontal) binocular optics and holographic optical elements 
for high brightness and transmissivity (see Fig. 2). Two high- resolution 1280x1024 CRTs were used as the image 
sources. The HMD weighed approximately 6.5 pounds and could be worn by most pilots for over an hour without 
discomfort. The HMD was driven by a graphics workstation at 1280 horizontal by 1024 vertical resolution and 
updated at a 60 Hz non-interlaced rate. A Polhemus magnetic head tracking system was utilized. 

A piloted simulation study was conducted to determine whether attitude information (pitch ladder, velocity 
vector symbol, and waterline symbol) displayed in a HMD should be presented with respect to the real world 
(conformal) or to the aircraft (body axis) for spatial awareness in a fighter aircraft. With the conformal presen- 
tation, the appearance of the displayed information was dependent on the pilot’s head position. The horizon line 
would always overlay the horizon of the outside scene, if it was in view; however, the attitude of the aircraft (nose 
position) could not always be easily obtained unless the pilot’s line-of-sight was aligned with the aircraft’s body 
axis. With the body-axis concept, the information was displayed as if the pilot was always looking directly out 
of the front of the aircraft, no matter which direction the pilot moved his head. This concept was analogous to 
physically mounting a Head-Up Display (HUD) to the helmet. Although the pilot could directly determine the 
aircraft’s attitude, in situations where the pilot’s line-of-sight was not aligned with the aircraft’s body-axis, the 
horizon line, if in view, would not overlay the horizon of the outside scene. The two display concepts were eval- 
uated using simulated air-to-air intercept tasks where the pilot was to obtain a gun solution on a maneuvering, 
but not interactive, target. 

The quantitative results favored the body-axis concept. Although, no statistically significant results were 
found for either the pilots’ understanding of roll attitude or target position, pitch judgment errors were made 
three times more often with the conformal display. The subjective results showed the body-axis display did not 
cause attitude confusion, a prior concern with this display. In the post test comments, the pilots overwhelmingly 
selected the body-axis display as the display of choice. The pilots stated that the conformal display was hard to 
interpret and confusing because of the symbology motion caused by the aircraft and head movements. However, 
the pilots commented they were more familiar with the body-axis display format because they used HUDs. With 
more training, the conformal display may have been more useful to the pilots. 

3.1.2 Microvision HMD 

In 2004, research was conducted using a military style helmet (see Fig. 3) for Synthetic Vision System (SVS) 
flight deck concepts. 10 The HMD was part of the Virtual Cockpit Optimization Program (V-COP) effort under 
the US Army and using the Microvision virtual retinal display concepts. The HMD was full color, fully binocular, 
fully overlapped, 1280x1024 pixel resolution display. The Asension LaserBird was used for head tracking (see 
Fig. 10). The optical performance data showed some outstanding characteristics although the system suffered 
from reliability issues typical of many prototype technologies. 11 

NASA’s use with this clearly military display focused on the potential for a helmet display with an unlimited 
field-of-regard to greatly increase pilot situational awareness (SA) both in flight and on the surface. The data 
showed the value of off-boresight information in commercial operations, especially for surface operations; however, 
the data also clearly indicated that a large, military style helmet would be a non-starter for commercial crews. 
Pilots expressed a desire for a light-weight, sunglasses form-factor type display (see concept in Fig. 4). 

3.2 Why an HWD for commercial and business aircraft? 

From this military heritage, several factors emerged that created a confluence of needs and capabilities for HWD 
research at LaRC targeting commercial and business aircraft flight decks. These factors were: a) consumer 
displays technologies; b) head-up displays and safety of flight; and, c) vision system technologies. 





Figure 5. Lumus DK-32 display glasses coupled with 
Figure 4. Concept picture of a light-weight, sunglasses a prototype inside-out head tracker made by Thales 

form factor HWD for commercial aviation use. Visionix. 

3.3 Synthetic and enhanced vision system 

Starting in the early 1970s, vision system technologies - Synthetic Vision (SV) and Enhanced Vision System 
(EVS) and related instantiations - were being researched by NASA. EVS technologies were maturing rapidly as 
evidenced by a large FAA flight test, 12 with NASA and others, 13 highlighting the operational potential of these 
systems. Vision systems technologies - SV/EVS - creates an electronic means of visibility for the flight crew, 
independent of the prevailing natural lighting or atmospheric conditions. 

Vision systems technologies moved to the forefront of NASA’s aeronautics mission directorate as part of a 
project to develop and deploy SV systems technologies, as a complement to EVS, to mitigate the leading cause of 
commercial aviation accidents world-wide, Controlled Flight Into Terrain (CFIT). 14 The focus of this program 
was to get these technologies into existing flight decks in order to have the greatest impact on CFIT accident 
reduction. 

In 2004, the concept of vision systems technologies took-off when the FAA amended the operating regulations 
for takeoff and landing under Instrument Flight Rules (IFR) contained in Section 14 of the Code of Federal 
Regulations, §91.175 to allow the use of an Enhanced Flight Vision System (EFVS). This rule change set an 
important precedence as the first operational credit provided to an imaging sensor system, creating an allowable 
electronic means of vision for the flight crew. 

Similarly, the fundamental technologies for SV - a computer-generated rendering of stored terrain topography 
from the perspective of the pilot - were beginning to emerge at program launch. In fact, when the program started, 
it was felt by many as being too revolutionary. The state of the art computer at program launch (circa 1997) 
was a 266 MHz Pentium Processor, 32 Mbytes RAM, 5 Gbyte Hard-drive. Fortunately, by the program’s end 
in 2007, the three key technologies - Global Positioning System (GPS), computer graphics rendering and flash 
memory were common-place and the requisite technologies could be found in everyone’s cell phone today. SV 
is now seen as the baseline standard in flight deck designs. 

Today, vision system technologies and the concept of an electronic means of visibility for the flight crew, 
independent of the prevailing natural lighting or atmospheric conditions, is a critical piece of the NextGen 
architecture. 

3.4 HUD 

HUDs have been available on commercial and business aircraft for many years and by the mid 1990s, was be- 
coming an accepted flight deck staple, although still not universally adopted. The HUD has proven itself as a 
valuable addition to the flight deck providing many safety and operational benefits. The advantages of HUDs 
for commercial aircraft is primarily the situational awareness enhancement. The HUD allows an “eyes-out” 
conformal view of the outside world without the requirement to go “heads-down” to look at flight instruments. 
The conformal display of attitude, flight path, and energy management information is key. The Flight Safety 




Foundation (FSF) 15,16 concluded that Head- up Guidance System Technology would likely have positively influ- 
enced the outcome of hundreds of accidents included in a study of turbine-powered, modern glass cockpit aircraft 
accidents. 

These HUD capabilities became an integral part of the EFVS rule making. The HUD is the only display 
currently certified and approved for use as an EFVS. The operating paradigm of the EFVS is the conformal 
display of imaging sensor data on the HUD, with conformal flight symbology overlaying the real-world, when 
natural vision is available. The EFVS operational credit (as per §91.175 (1) and (m)) explicitly expressed that 
the use of a HUD was an essential “characteristic and feature” of the EFVS operation. 

In developing the new rule, the FAA recognized emerging technologies and placed within the rule, provisions 
for the use an “equivalent display.” 

3.5 The HWD for commercial aviation 

As NASA and industry were maturing HUD and vision systems technologies, the small display form-factor that 
the pilots desired was just emerging. The technologies for a viable HWD on a commercial flight deck were 
approaching. 

LaRC’s HWD research with this emerging technology was primarily driven by two operational paradigms: 

1. The HWD can provide unlimited field-of-regard; hence, the HWD may provide operational enhancements 
(safety and/or efficiency) which can take advantage of this capability especially since there are now vision 
systems technologies that are unlimited in field of regard and can be effective independent of the prevailing 
natural visibility. 

2. The HWD may be a “HUD equivalent.” In this scenario, the HWD can be advantageous where: a) HUD 
installation is not possible or practical by volume or weight; b) HUD retro-fit is not cost-effective; and, c) 
HWD installation has a return-on-investment advantage to the HUD. 

The ‘easy’ part of the research was defining the HWD requirements for these operational paradigms. The 
requirements are simply: 

1. Equivalent optical performance as today’s HUD (as defined in such documents as SAE ARP-8105). 

2. No more encumbrance to the pilot as that provided by today’s head- worn devices. 

The extremely challenging part of this task is to meet these two requirements. These requirements are 
diabolically opposed: 

• State-of-the-art HUDs are fantastic optical devices, with outstanding clarity, transmissivity, and - by design 
and also because they are firmly attached to large aircraft - outstanding symbolic and image stability. In 
a recent EFVS flight test, the end-to-end latency in an EFVS presentation on a HUD was measured to be 
greater than 200 msec due to the combination sensor, image processing and HUD processing. 17 Nonetheless, 
each of the test pilots found the presentation to be excellent and latency was never once considered an 
adverse factor. 

• A rough summary showed a variety of head-sets that pilots sometimes, but not always wore. The type, size, 
and function varied. The one item of head-worn gear that almost all pilots wore was sunglasses (with the 
obvious exception of night operation). As such, the goal was set to sun-glasses’ “equivalent encumbrance.” 





Figure 6. Monochrome, monocular HWD circa 2010, 4 Figure 7. Intersense prototype tracker developed for 

ounces. NASA Langley. 


3.6 Head-tracking technology 

To meet the operational paradigms and their associated performance requirements, the HWD must be coupled 
with a high performance head-tracker. The tracker, while being lightweight and unobtrusive, must meet HUD-like 
static and dynamic accuracy requirements. 

Under the Small Business Innovative Research (SBIR) program, NASA Langley awarded a contract to Inter- 
sense, Inc. (now Thales Visionix) to develop a head tracker to meet these performance requirements and that 
the HWD system be minimal encumbrance. 

Research examined these innovative augmented reality approaches to HWDs for commercial surface opera- 
tions. 18 

Intersense delivered two prototype head trackers under the SBIR contract. The first tracker was a prototype 
based on their IS- 1200 VisTracker that is an inertial tracker where the inertial drift is corrected by image 
processing (see Fig. 7). The focus of the prototype was miniaturization of the tracker by utilizing a cell phone 
camera. Though small in size, the tracker used visible light detection of circular bar codes, which under certain 
lighting conditions, could adversely affect the performance of the tracker. 

To improve upon the short comings of the first, tracker, Intersense delivered an infrared based camera thus 
solving the visible light issue of the previous tracker but at a cost of a larger tracker (see Fig. 5). This second 
tracker was also inertial with optical tracking and image processing to correct the drift. 

3.7 System latency measurement 

Head-tracker latency dictates the dynamic symbol/imagery positioning accuracy. An end-to-end latency require- 
ment of no more than 20 milliseconds has been proposed for Virtual HUD applications based on previous work. 19 
However, the acceptable latency may become significantly smaller if dynamic stability is a driving requirement. 
The SAE AS8055 document, the “Minimum Performance Standard for Airborne Head Up Display (HUD),” 
standards suggest that this is the case. 

The technical challenge is that this allowable latency is an ‘end-to-end’ requirement. A basic HWD with head 
tracking system, from end-to-end, is comprised of: 1) a near-to-eye display, 2) the head tracking system, 3) one 
or more symbology or image sources, 4) and the display/image processor. Each element and the communication 
between them contribute a portion to the total latency. No commercial or standardized device is available to 
measure and quantify end-to-end latency. NASA has developed a prototype Head Mounted Display Latency 
Measurement Rig (HeLMR) for this purpose. 19 The HeLMR apparatus consists of an anatomically correct, 





Figure 8. HeLMR System (Head, Camera, Rotary Stage, Stage Controller, DC Light Source). 


human head which is able to ‘wear’ available commercial and custom HMD systems (see Fig. 8). A camera is 
installed in place of the eye(s) in the correct image plane location. The head is mounted on a precision rotary 
stage that moves the head in a left-right-lcft ‘No-No’ fashion at a precise angular rate. To measure the end- 
to-end latency, a space-stabilized symbol is rendered on the HWD along the boresight. As the head is slewed, 
the space-stabilized symbology becomes misaligned with respect to the outside reference, proportional to the 
end-to-end system latency and the head form angular rate. 

4. HUMAN-IN-THE-LOOP HWD RESEARCH 

4.1 Monocular, biocular and binocular displays 

HWD may be monocular, binocular, or bi-ocular in design and each have their advantages and disadvantages and 
individually impart different human factors concerns. For example, binocular rivalry and disparity are known 
physiological issues caused by HWD design and affect perception. And other human factor design issues involving 
rotational, magnification, and luminance difference between the eyes significantly minimizes the display potential. 
Double imagery, adaptation, motion effects, and many other human factors concerns can impact not only the 
perception of the imagery, but also the cognition of the user (e.g., effects on decision-making, response time, 
accuracy, judgments, change blindness, cognitive tunneling, visual search, memory, problem-solving, situation 
awareness, attention, etc.) as well as physiological consequences due to the perceptual and/or cognitive mismatch 
(e.g., eye strain, disorientation, headaches, and sickness). Each of these issues are addressed in many ways 
throughout the existing HMD literature but much of this existing literature is also heavily biased toward the 
military application or academia and its youthful subject population. Our application, conversely, has to consider 
pilots up to the age of 65, if not older, with a variety of vision conditions, corrections, and color-deficiencies. 
HWD design considerations, especially that of the use of color and the optical design, as they interact with the 
HWD user age, are somewhat unique in this new application of HWDs. 

Honeywell performed research for NASA to examine issues with monocular versus biocular HWD displays. 4 
Honeywell utilized Microvision Nomad displays with an Ascension Phasor Bird head tracker to create the monoc- 
ular and biocular HWD display conditions. The three display conditions examined: monocular display on the 
dominate eye, monocular display on the the non-dominate eye and a biocular display. Results showed no sig- 
nificant differences in flight performance between the three display concepts. An interesting result from the 
Honeywell study was that the monocular display was significantly more accurate in terms of visual acuity com- 
pared to a biocular display. No binocular rivalry effects were found in this seasoned group of aviators. Thus, 
NASA began surface operations research with a simpler, light-weight monocular HWD. 

4.2 Taxi operations 

Our research showed that one operational area where off-boresight information for commercial and business 
transport aircraft was critical was in surface operations. Further, at the beginning of the 21 st century, the 



National Transportation Safety Board (NTSB) continually included runway incursion prevention on its “top 
most wanted” list for aviation safety. 20 The application of HWD technology to address this surface operations 
was born. 

On the flight deck, the available out-the-window visibility provides the ‘truth’ and at least one pilot is always 
head-out during the flare and during surface operations. The HUD or HWD must be designed to not obscure 
this outside view. Today, HUDs are certified only after demonstrating that they are in compliance with Part 
14 of the Code of Federal Regulations §25.773, showing that the HUD design still gives the pilots “a sufficiently 
extensive, clear, and undistortcd view, to enable them to safely perform any maneuvers within the operating 
limitations of the airplane, including taxiing takeoff, approach, and landing.” The HUD or HWD should ideally 
augment the prevailing visibility - providing sufficient information to enhance or enable the operation - without 
significantly obscuring it. 

Full-color HWD display concepts were evaluated in surface operations in direct or indirect comparison using 
previous research against HUD equipage for taxi route awareness, traffic awareness, taxi efficiency, and runway 
incursion prevention. 



Figure 9. Full color, monocular HWD circa 2007, 2 
pounds. Outside-in laser head tracking. 



Figure 10. Ascension LaserBird head tracker. Three 
prong sensor mounted on the back of the HMD shown 
in Fig. 9. 


4.2.1 HWD down-select usability study 

A HWD usability study was used to down select concepts for feasibility. 21-23 The usability study was conducted 
in the medium fidelty, fixed-based Visual Imaging Simulator for Transport Aircraft Systems (VISTAS) simulator. 
The usability study was designed to demonstrate the efficacy of a head-worn display which provides unlimited 
field-of-regard Synthetic Vision for surface operations. The results demonstrated that providing pilots with 
the ability to virtually see well beyond visual range can significantly increase situation awareness and task 
performance on the airport surface. Pilots were better able to perform the taxiing evaluation task and reported 
significantly higher situational awareness with the HWD concepts compared to an electronic moving map or 
paper charts of the airport environment. Furthermore, the study provided tremendous insight into future design 
and development of head- worn displays, including hardware considerations and methods for integration of display 
modes. 

The usability study highlighted two significant hardware considerations. Nearly all pilots rated the higher 
resolution (800x600), see-through FIWD over the lower resolution (640x480), opaque HWD. Pilots commented 
that the higher resolution improved the readability of the display especially for text and numbers. Additionally, 



the pilots preferred not to have their forward vision blocked; even by the small 640x480 pixel display. The 
see-through capability allowed the pilots to continue their nominal out the window surveillance of the airport 
environment during taxi. Also, the see-through display provided the pilots with confidence that the display was 
aligned with the scene. For surface operations, it is important for a HWD to be see-through because, for all 
practical purposes, the HWD will always be providing an “augmented reality” not a “virtual reality” condition. 

4.2.2 High-fidelity simulation experiments 

From the results of the usability study, two experimental studies were conducted to determine the efficacy of using 
HWDs to enhance taxi operations. 24,25 For both experiments, full-color HWD display concepts were evaluated 
in surface operations to address previously witnessed display technology limitations. 

The experiments were conducted in NASA’s high fidelity simulator known as the Research Flight Deck (RFD). 
The RFD was equipped with a 30° horizontal x 24° vertical HUD on the captain’s side. The HWD, worn only 
by the captain, was a Liteye LE-500 800x600 pixel, full color display with see-through capability (Fig. 9). The 
head tracker was laserBIRD tracker by Ascension Technology Corporation. A skateboard helmet was used as 
the mounting location for the display and tracker. The helmet was not grossly heavy but was sturdy enough 
with good fit and comfort for good stability. It also had some acceptance from an aesthetic viewpoint. 

The pilots placed the display just above their right eye so that it was visible by glancing up which maintained 
unimpeded stereoscopic vision for out-the-window monitoring. The resulting display was conformal to the real- 
world (out-the- window scene) if the pilot tilted his or her head down. This “semi-conformal” presentation allowed 
pilots to bring the display into view when they desired. 

The pilots conducted simulated taxi operations at Chicago O’Harc International Airport. The display condi- 
tions were varied by having no HUD, a HWD or a traditional HUD. A head-down Electronic Moving Map (EMM) 
display was varied from a basic moving map to an advanced moving map which included routing and other traffic. 
The display condition and weather were experimentally varied. A total of 27 different taxi scenarios were used in 
the study. Three of the 27 scenarios were “rare-events” to test off-nominal, safety-critical stress cases. All taxi- 
ing tasks involved exiting the active runway and taxiing to the airport movement area boundary. The weather 
state for the out-the-window scene was varied between night-time with unlimited visibility Visual Meteorological 
Conditions (VMC), and daytime with 700 Runway Visual Range (RVR). 

From the post-test analysis for this experiment, there were no significant differences between the HUD and 
HWD concepts. 

A second experiment was designed to be a follow-on to the experiment mentioned above to improve the color 
of the displayed traffic and to evaluate additional display concepts to the baseline condition of an airport paper 
map in order to fully complete the matrix of possible display comparisons. From the first experiment, even 
though traffic was displayed, some crews missed the displayed traffic and ended up in a “nose-to-nose” situation 
for the rare event scenario. For the second experiment, none of the crews who had traffic displayed got into a 
nose-to-nose situation and, therefore, it appears likely that the color and size of the traffic icon were the main 
factors for the differing results from the first experiment. 

The performance data showed that better route accuracy and faster taxi speeds can be obtained using the 
HWD and HUD compared to paper charts alone. On average, the pilots were able to complete the taxi route 
15% faster with the HWD and HUD concepts compared to paper charts. 

From both experiments, the results with the HUD were similar to the results from previous surface operations 
research conducted by NASA Ames and NASA Langley. No quantitative performance differences differentiating 
head-up versus head-down display concepts (when displaying essentially the same or similar information) were 
found in these studies. Additionally, the crews made significantly more navigation errors with the paper charts 
than with any of the other three advanced display concepts. Comparing the HWD and HUD concepts across 
both experiments, there were no display effects, visibility effects, or interaction effects of these two main factors 
for the three measures listed above. Therefore, in terms of taxi performance, the HWD and the HUD were 
statistically the same. 



H350 



Figure 11. Surface concept with the HWD. 


4.2.3 Augmented reality surface operations 

Instead of using a ‘semi- conformal’ display, an alternate approach was developed to augment the scene and not 
obscure the prevailing natural visibility. Figure 11 graphically depicts an instantiation of the augmented reality 
HWD concept for surface operations. The boxed area containing symbology and the virtual airport represents 
the view as it would be rendered on the HWD. Outside the boxed area represents the visibility of the actual 
airport environment with natural vision. In this example, the figure represents a reported visibility of 700 RVR. 
The augmented reality concept draws the S/EV imagery (i.e. the ’virtual airport environment’), beyond the 
reported 700 RVR, which represents that “Beyond RVR.” Note that the SV airport is not shown in the HWD up 
to 700 RVR as this portion of the actual airport can be seen with the unaided eye. Essentially, the SV airport 
was culled up to 700 RVR to allow pilots to view the actual airport environment. The cleared route (shown as 
the magenta ribbon) was drawn on top of the yellow taxiway centerline to denote the cleared path. The cleared 
path always overlays the taxiway centerline if in view on the HWD as it is critical information for runway and 
taxiway incursion prevention. Traffic, within the conformal view, are depicted with the augmented reality traffic 
diamond. 

Pilots were asked to taxi complex routes, under simulated low-visibility conditions (300RVR, 600RVR, 
2400RVR), in a large commercial transport aircraft simulator at Chicago O’Hare (FAA identifier: ORD). The 
test conditions evaluated the “Beyond-RVR” concept and augmented reality traffic depictions compared to the 
virtual reality presentations used in prior studies. The augmented reality concept rendered virtual airport ob- 
jects (such as runways, t.axiways, centerlines, tower(s), buildings) obscured by the weather in the HWD. In other 
words, the HWD virtually displays only airport objects and traffic that cannot be seen naturally (i.e., beyond 
the reported runway visual range) and thus, not obscuring naturally visible objects. If the object was neared 
than the RVR value (i.e., 300RVR, 600RVR, or 2400RVR), then the “curtain,” drawn at the reported RVR, 
would allow pilots to see naturally that scene. Beyond the reported RVR, the visibility curtain would obscure 
the natural scene and instead a synthetic scene and real-time imagery (e.g., Forward Looking Infrared (FLIR)) 
would conformally depict the world beyond that distance. 

The results of the research evinced that the HWD, regardless of concepts tested, significantly enhanced 
situation awareness compared to more traditional displays used during aircraft taxi. Although no quantitative 
differences were found, the usefulness of the Beyond-RVR concept had merit only at higher visibilities. At very 
low- visibility operations, there were no discernable difference to the pilots between the concepts tested (i.e., 
the 300RVR and 600RVR condition practically looked the same regardless of display used during taxi). It was 
further observed that higher fidelity simulator or real-world aircraft would increase the differences as the aircraft 
simulator used did not emulate actual conditions as well. Pilot comments supported the value of Better than 
Visual (BTV) to increase capacity and safety in the airport movement area and that innovative approaches for 
intuitive display of Synthetic and Enhanced Vision (S/EV) have merit and research should continue. 

The results from these surface operations with HWDs led to NASA patenting the technology. 26 Follow-on 
experiments continue to refine surface operations research with HWDs. 



4.3 Simultaneous Offset Instrument Approaches using a HWD 

A significant hurdle to overcoming capacity constraints are simultaneous dependent parallel runway operations 
at runways separated by less than 4300 feet. In particular, very closely spaced parallel runways have equipment 
and procedural requirements which significantly limit their use in degraded visual conditions. When the weather 
drops below visibility or ceiling minima, an airport is reduced to single runway operations substantially reducing 
arrival rates (from 65 to 30 aircraft per hour at San Francisco (SFO) airport). 

Simultaneous dependent parallel runway operations require a pilot to see-and-avoid parallel traffic. To meet 
the “weather-independence” goal of NextGen, game-changing technology is needed to safely overcome these 
constraints. Arthur, Prinzel, et al. (2009) 25 reported on representative study that demonstrated the usefulness 
of HWDs, paired with other technologies, to more effectively address the problem prescribed for NextGen. The 
research examined the use of a S/EV HWD concept, in concert with Flight deck Interval Management (FIM) 
technologies, to conduct very closely spaced simultaneous dependent parallel runway operations under restricted 
visual conditions. The simulation experiment evaluated head-down and head-up displays (both HUD and HWD), 
paired with vision systems and FIM technologies, to conduct these dependent arrival operations. 

In general, pilots rated using a HWD in Instrument Meteorological Conditions (IMC) the same as the visual 
approach (unlimited visibility) in terms of situation awareness and mental workload. The results were based 
on some simplifying technological assumptions but nonetheless, provide an outlook of the potential of HWDs 
to serve as an enabling technology and flight deck display platform to facilitate the envisioned path toward the 
goals set forth by NextGen and future air transportation systems. 

4.4 HWDs for spacecraft 

In 2007, NASA began exploring technology for a human-mission to the moon known as the Constellation Pro- 
gram. 2 ' The lunar lander spacecraft, known as Altair, was to have significantly reduced out-the- window look- 
down angle compared to the Apollo lunar lander. 28,29 A part-task simulation was conducted to explore using 
a monocular HWD (see Fig. 6) for a lunar lander spacecraft. 30,31 During Apollo, the constraints placed by the 
design of the Lunar Module (LM) window for crew visibility and landing trajectory were “a major problem.” 
Optimal fuel saving trajectories render the natural vision of the crew from windows inadequate for the approach 
and landing task. Thus, a light-weight HWD system with SV and perhaps Enhanced Vision (EV) technologies 
offered a potential solution for spacecraft crews. 

In general, pilots thought the HWD had great potential but was not optimized for this lunar landing task. The 
performance data shows there were no significant performance differences when using the HWD in conjunction 
with the head-down displays (HDDs). Also, there were no significant workload or simulator sickness effects with 
the HWD. Regarding SA, the data is anecdotal. Some pilots felt the HWD provided greater SA because the 
HWD allowed for an eyes-out view while still being able to perform the task. Pilots preferred this eyes-out view 
as it provided “truth” data as to what the actual situation is, rather than relying on a computed navigation 
solution which can be subject to errors. These comments were from pilots who were familiar with and frequently 
flew night vision systems which are monocular and monochrome green; therefore they were used to and familiar 
with having a monocular display over one eye while flying. 

However, those pilots who had not flown such systems were distracted by the HWD at times. The artifacts 
associated with a head tracked display, such as blurry text and numbers, caused pilots to abandon using the HWD 
and rely solely on the HDDs. Thus, pilots would try to keep their heads still in order to reduce latency effects 
which reduced the readability of the HWD. All pilots agreed that a HWD with a larger ficld-of-view (FOV) 
would be desirable; however, pilots were not asked to quantify how much larger the FOV should be. 

Binocular rivalry effects were not observed with the HWD system used in this experiment. Binocular rivalry is 
a well documented phenomena where two vastly different images are presented to each eye. When two disparate 
images are presented to each eye, the brain involuntarily suppresses one of the images. Even though a monocular 
HWD system was used in this experiment, binocular rivalry was not so much an issue as the transparent HWD 
allows for similar images to the eyes. 32 

With head tracked HWD systems, many factors can affect the image quality, thus the acceptability of the 
HWD. 33 In this experiment, synthetic terrain was rendered on the HWD. The alignment of the synthetic 



terrain to the real terrain (out-the-window is considered truth for this experiment) is dependent on the static 
boresighting as well as the system latency. Latency will cause an apparent misalignment during head movement, 
but as the pilot’s head comes to a stop, the terrain will appear to “catch-up.” This terrain “swimming” can lead 
to simulation sickness and loss of confidence in the fidelity of the system. For this experiment, neither simulation 
sickness nor integrity of the HWD system was a concern. 

4.5 HWD as an equivalent HUD 

As mentioned, the FSF identified significant safety benefits of head- up/HUD flight operations. 15 In addition to 
safety benefits, “operational credits” are now being derived from HUD equipage that a HWD might also obtain 
if “HUD equivalence” can be shown. In particular, the EFVS operational credit (as per §91.175 (1) and (m)) 
explicitly expressed that the use of a HUD was an essential “characteristic and feature” of the EFVS operation. 
Two tests were conducted to assess the state of the HWD technologies to meet the provisions for the use of a 
HWD as an equivalent display to the HUD. 

4.5.1 Simulation experiment 

Under the Vehicle Systems Safety Technologies (VSST) project in the Aviation Safety Program (AvSP), one 
specific area of research is the use of small HWDs (see Fig. 5) as an equivalent display to a HUD. 16 A simulation 
experiment was conducted to evaluate if the HWD, coupled with a head-tracker, can provide an equivalent 
display to a HUD. Comparative testing was performed in the RFD Cockpit Motion Facility (CMF) full mission, 
motion-based simulator at NASA Langley. Twelve airline crews conducted approach and landing, taxi, and 
departure operations during low visibility operations (1000’ RVR, 300’ RVR) at Memphis International Airport 
(Federal Aviation Administration (FAA) identifier: KMEM). 

The HWD used in this experiment was coupled with a prototype head tracker that provided head orientation 
and was mounted on the left side of a pair of Lumus DK-32 glasses. The Lumus eye-wear is see-through, full 
color which utilizes patented Light-guide Optical Element (LOE) technology to generate an image that appears 
at “practical” infinity similar to a HUD. For this experiment, only monochrome green symbology and imagery 
were displayed on the HWD as to not introduce a confound when comparing to the monochrome HUD. 

The results showed that there were no statistical differences in the crews performance in terms of touchdown 
and takeoff. Further, there were no statistical differences between the HUD and HWD in pilots’ responses to 
questionnaires. 

4.5.2 Flight demonstration 

Using the same HWD system described above, a flight demonstration was conducted at NASA LaRC. 34 The 
purpose of the flight test was primarily to evaluate the use of HWDs during actual aircraft taxi and approach 
operations. Approach and taxi testing was performed on board NASA Langley’s experimental King Air aircraft in 
both VMC and simulated IMC conditions. Seven highly experienced test pilots with HUD experience participated 
in the flight test. Since NASA’s King Air did not have a HUD installed, pilots were asked to compare the HWD, 
in which they used to fly the airplane, with a HUD based on previous experience. 

The pilots flew straight-in Instrument Landing System (ILS) approaches wearing the HWD. The HWD 
symbology consisted of a ‘virtual-HUD’ concept where a typical HUD symbology was rendered if the pilot was 
looking at the area where a HUD combiner glass would be mounted. In addition to typical flight symbology, a 
simulated FLIR image was rendered on the HWD and was conformal to the outside world. During simulated 
IMC, the pilots view through the HWD was blocked leaving only symbology and simulated FLIR imagery (FLIR 
on/off was experimentally controlled). 

Pilots were able to fly the approaches and stay within a dot of precision on the localizer and glideslope using 
the HWD. Pilot comments showed acceptance of the concept though provided comments to improve the HWD 
system. The most requested improvement was to stabilize the symbology which essentially translates to reducing 
the system latency. 

When pilots flew simulated IMC, the FLIR imagery was not misaligned with the “real world” since it was 
not visible, thus, no eye strain even on turbulent approaches was reported. Pilots reported that this configuration 



reduced workload significantly and pilots commented they were able to focus on pertinent information much 
easier. For pilots that encountered light to moderate turbulence, system latency created a “jittery, bouncy” 
display that was difficult to read and follow. The latency combined with turbulence resulted in eye strain 
and headaches (although minimal) causing increased workload. This combination of turbulence and latency is 
manifested in temporary discrepancies in the conformal image to the actual image. 

5. FUTURE DIRECTIONS 

The work to date indicates that a Head- Worn Display for commercial and business aircraft is viable. The data 
also suggests that the business case to make this happen is through the path of “HUD equivalency.” The data 
also shows that the technology is not ready quite yet. The form factor and static and dynamic accuracies are 
not where they need to be for HUD equivalence. The other open question is how to best meet the challenge of 
obscuration and contamination of the pilot’s view outside the aircraft when there is prevailing natural vision, 
such as surface operations. 

The application domain for this technology lays strongly in three areas: a) HUD installation is not possible or 
practical by volume or weight; b) HUD retro-fit is not cost-effective; and, c) HWD installation has a return-on- 
investment advantage to the HUD, such as a weight reduction. Once installed, the HWD-equipped aircraft can 
then pursue HUD operational credits for reduced operating minima for landing and take-off. More importantly, 
if the HWD can get installed as a HUD equivalent, the data suggests that several HWD-unique applications 
will open up. Improvements have been shown for surface operations and in flight operations where off-boresight 
and expanded field-of-vicw information for traffic identification and extended runway centerline awareness is 
important. 

The other “game-changing” application could be enabling an emerging NextGen concept termed, Equivalent 
Visual Operations (EVO). EVO is an electronic means to provide sufficient visibility of the external world 
and other required flight references that enable the safety, operational tempos, and VMC-like procedures for all 
weather conditions. The HWD, coupled with SV and EVS technologies, would create an intuitive interface or an 
electronic Visual Flight Rules (E-VFR) operational capability. A step further, EVO would be one component of a 
“better-than- visual” operational capability; replicating the capacity of today’s VMC flight and more importantly, 
meeting and improving on the safety of today’s VMC flight in all-weather NextGen operations. NASA research 
aims to extend the present-day S/EV concepts using HWDs to enable VMC-like operational tempos and maintain 
and improve the safety of VMC while using VMC-like procedures in all-weather conditions. 
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